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Abstract 


The  purpose  of  the  research  project  was  to  understand  the  future  crew 
environments  for  developing  unmanned  aerial  vehicle  (UAV)  systems.  A 
variety  of  human  engineering  tools  (job  assessment  software  system  [JASS], 
enhanced  computer-aided  testing  [ECAT],  and  MicroSaint™)  were  used  to 
address  crew  issues  related  to  the  utility  of  having  rated  aviators  as  crew 
members,  supplementing  current  crews  with  imagery  and  intelligence 
specialists,  and  the  use  of  automation  to  improve  systems  efficiency.  Data 
from  70  soldiers  and  experts  from  Fort  Huachuca,  Arizona,  Fort  Hood,  Texas, 
and  Hondo,  Texas,  were  collected  as  part  of  this  effort.  The  general  finding 
was  that  the  use  of  cognitive  methods  and  computerized  tool  sets  to 
understand  future  crew  environments  proved  to  be  cost  effective  and  useful. 
Specifically,  no  evidence  was  found  to  support  a  requirement  for  rated 
aviators  in  future  Army  missions,  but  the  use  of  cognitively  oriented 
embedded  training  simulators  was  suggested  to  aid  novices  in  developing  the 
cognitive  skills  evinced  by  experts.  The  efficacy  of  adding  imagery  specialists 
to  96U  crews  was  discussed,  and  specific  recommendations  related  to 
automation  were  derived  from  the  workload  modeling. 
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CREW  SYSTEMS  ANALYSIS  OF  UNMANNED  AERIAL  VEHICLE  (UAV) 
FUTURE  JOB  AND  TASK  ENVIRONMENTS 


INTRODUCTION 

Future  battle  spaces  will  be  exploited  by  a  variety  of  aerial  and  ground  platforms  to  help 
U.S.  forces  achieve  information  dominance.  The  unmanned  aerial  vehicle  (UAV)  family  of 
platforms  will  increase  the  range,  survivability,  and  effectiveness  of  this  effort.  The  purpose  of 
this  ongoing  research  is  to  understand  the  crew  environment  and  soldier  performance  issues 
related  to  future  UAV  systems.  Specifically,  three  major  issues  were  addressed:  (a)  the 
importance  of  using  rated  aviators  for  piloting  positions  for  the  UAV,  (b)  the  use  of  imagery 
specialists  and  intelligence  analysts  (96D  and  96B  military  occupational  specialty  [MOS])  as 
adjunct  crew  members,  and  (c)  the  potential  use  of  automation  to  assist  in  future  crew  functions. 
The  variety  of  questions  asked  required  the  use  of  numerous  human  engineering  and  human 
performance  data  collection  methods. 

A  secondary  motivation  was  to  investigate  the  effectiveness  of  available  tool  sets  and 
methodologies  to  understand  human  job  and  mission  environments  for  developing  systems.  The 
best  way  to  test  the  mettle  of  these  approaches  was  by  attacking  important  problems  of  developing 
systems  rather  than  by  investigating  laboratory  problems  of  dubious  validity.  The  UAV  was  an 
ideal  candidate  because  of  its  crew-intensive  mission  profiles  (Barnes  &  Matz,  1998)  and  the  need 
to  investigate  the  crew  issues  perceived  by  the  Training  and  Doctrine  System  Manager  (TSM). 

The  TSM’s  cooperation  was  absolutely  essential  in  completing  this  study;  in  providing  direction, 
expertise,  and  a  sense  of  priorities;  and  because  a  significant  investment  of  the  TSM  personnel’s 
own  time  and  effort  was  required  during  the  data  collection  and  analysis  portions.  The  overall 
study  was  extensive,  including  the  efforts  of  more  than  70  participants  representing  expertise  from 
the  aviation,  intelligence,  and  UAV  communities  of  Fort  Huachuca,  Arizona,  Fort  Hood,  Texas, 
and  the  contractors  in  Hondo,  Texas,  who  engineered  the  Outrider  UAV. 


RATED  AVIATORS 

The  use  of  rated  aviators  as  part  of  the  UAV  crew  was  deemed  the  most  crucial  issue 
addressed.  The  problem  is  complicated  because  of  the  safety,  training,  and  selection  issues 
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involved.  In  the  UAV  crew,  two  flight  positions  were  examined:  the  internal  pilot  (designated 
air  vehicle  operator  [A  VO])  and  the  external  pilot  (EP).  The  A  VO  for  current  Hunter  UAV 
configuration  resides  in  the  ground  control  station  (GCS)  seated  next  to  the  mission  payload 
operator  (MPO).  The  AVO  coordinates  with  the  mission  commander  to  do  mission  planning, 
assumes  flight  control  of  the  air  vehicle  after  take-off,  and  sets  the  course  to  the  various  way 
points.  The  AVO  must  be  able  to  read  flight  instruments  and  understand  the  current  flight  status 
but  does  not  usually  fly  the  air  vehicle  in  the  sense  that  a  fixed  or  rotary  wing  aviator  does. 
Instead,  the  AVO  directs  the  UAV  to  a  target  location  and  upon  arrival,  coordinates  with  the 
MPO  who  executes  the  best  search  pattern  over  the  target  area.  The  AVO  also  responds  to  flight 
emergencies  and  makes  course  changes  for  tactical  or  safety  reasons.  However,  most  of  the 
initial  “hands-on”  control  of  the  air  vehicle  is  done  by  the  EP  who  flies  the  UAV  during  take-off 
and  landing  using  a  control  device  similar  to  that  used  for  radio-controlled  model  airplanes.  It  is 
important  to  note  that  most  flight  safety  problems  occur  during  the  EP’s  watch;  this  is  not  a 
result  of  any  characteristic  of  the  EP;  rather,  it  reflects  the  dangers  associated  with  take-off  and 
landing  for  any  air  vehicle. 

Method 

Procedure 

Four  analyses  were  performed  to  determine  the  important  cognitive  skills  required 
for  the  AVO  and  EP  positions  and  to  relate  them  to  safety-of-flight  issues.  Although  data  were 
collected  for  all  flight  functions  for  both  categories,  the  main  focus  was  on  flight  functions 
clearly  related  to  air  vehicle  accidents  and  incidents. 

Using  the  job  assessment  software  system  (JASS),  the  authors  collected  ratings 
from  UAV  AVOs  and  EPs  regarding  the  importance  of  an  array  of  cognitive  skills  to  their  jobs 
and  tasks.  Data  were  collected  from  flight-rated  U.S.  Army  aviators  to  contrast  the  cognitive 
skills  they  reported  as  particularly  important  with  skills  reported  by  the  UAV  EPs. 

Subsequent  analysis  indicated  that  JASS  data  painted  an  incomplete  picture;  it 
became  obvious  that  more  was  needed  to  be  known  about  the  relationship  between  reported 
cognitive  skill  levels  and  actual  mishaps.  One  source  of  information  concerning  the  relationship 
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of  performance  and  skill  level  was  the  training  experiences  at  the  UAV  Flight  School  at  Fort 
Huachuca.  JASS  data  were  supplemented  with  enhanced  computer-aided  testing  (ECAT)  data 
from  a  pilot  study  collected  by  Hopson  (1995).  This  study  correlated  the  ECAT  scores  on  one- 
and  two-handed  tracking  scores  with  failure  rate  for  the  EP  training  course.  In  addition,  the 
UAV  flight  incident  report  results  were  compared  to  the  JASS  flight  tasks,  which  permitted  us 
to  focus  our  analysis  on  critical  flight  functions  (TSM,  1998). 

Finally,  data  interpretation  proved  to  be  a  difficult  problem.  Besides  the 
relationship  of  tasks  to  skill  levels,  there  were  operational,  programmatic,  and  experiential  issues 
as  well  as  similar  investigations  by  other  services  to  consider  in  attempting  to  forge  a  position  on 
rated  aviators  from  the  raw  data.  To  address  these  issues,  a  subject  matter  expert  (SME)  working 
session  was  convened  on  15  October  1998  at  Fort  Huachuca  in  order  to  help  interpret  the  data 
(see  list  of  participants  in  Appendix  A). 

Participants 

For  the  JASS  data  collection,  a  total  of  30  96U  soldiers  or  Hunter-trained 
contractors  was  tested  during  the  exercise.  There  were  21  MPO  and  A  VO  designations,  1 1  of 
whom  provided  JASS  data  from  a  primarily  A  VO  task  structure  and  10  from  a  primarily  MPO 
task  structure  for  this  part  of  the  data  analysis.  The  AVO  task  list  consisted  of  AVO  tasks 
associated  with  flight  and  navigation  functions,  excluding  tasks  involved  with  take-off  and 
landing.  In  addition,  nine  certified  EPs  were  tested  using  the  external  pilot  task  structure  for  the 
JASS  testing.  Further,  because  of  the  difference  in  EP  experience  levels,  those  with  a  year  or 
less  of  experience  were  considered  the  low  experience  group  (4)  and  those  with  more  than  1  year 
of  experience  (5)  were  designated  the  high  experience  group.  The  EP  task  list  consisted  of 
functions  related  to  take-off  and  landing  an  air  vehicle.  This  same  list  of  EP  functions  was 
administered  to  16  currently  rated  U.S.  Army  aviators.  The  aviators  characterized  themselves  as 
primarily  fixed  wing  (10)  or  rotary  wing  (6)  when  they  answered  JASS. 

Data  from  the  ECAT  selection  test  battery  were  used  in  this  analysis  as  well.  The 
two  sub-tests  used  were  the  one-  and  the  two-handed  tracking  tasks.  This  test  was  administered 
in  1995  and  used  a  sample  of  28  students  from  both  the  Pioneer  and  Hunter  external  pilot  classes 
held  at  Fort  Huachuca,  including  six  participants  who  failed  the  external  pilot  course.  Finally,  a 


5 


SME  group  consisting  of  12  members  was  used  to  help  interpret  the  data.  The  group  was 
comprised  of  military,  contractor,  and  civilian  personnel  with  operational  and  human  engineering 
backgrounds  related  to  the  UAV  crew  issues. 


Test  Instruments 

Job  Assessment  Software  System  (JASS) 

JASS  is  a  test  instrument  developed  to  elicit  from  soldiers  the  relative 
importance  of  50  skills  and  abilities  for  specific  task  functions  defining  various  MOSs.  The 
computerized  test  is  designed  to  allow  the  soldiers  to  rate  each  skill  designation  on  a  seven-point 
scale  for  each  specified  military  task.  The  itemized  skills  and  abilities  are  illustrated  in  Figure  1, 
broken  into  functional  areas:  communication,  speed-loaded,  reasoning,  visual,  auditory,  and 
psychomotor  (fine  and  gross  motor  skills).  The  test  is  based  on  validated  psychometric 
investigations  performed  by  Fleishman  and  his  colleagues  (Fleishman  &  Quaintance,  1984)  and 
broken  into  the  underlying  cognitive,  perceptual,  and  psychomotor  skills  that  would  constitute 
any  human  work  activity. 


Sensory 


Communication 


ORAL  COMPREHENSION 
WRITTEN  COMPREHENSION 
ORAL  EXPRESSION 
WRITTEN  EXPRESSION 


Perception 


Conceptual 

MEMORIZATION 
PROBLEM  SENSITIVITY 
ORIGINALITY 
FLUENCY  OF  IDEAS 
FLEXIBILITY  OF  CLOSURE 
SELECTIVE  ATTENTION 
SPATIAL  ORIENTATION 

visualization  Speed-Loaded 

TIME  SHARING  SPEED  OF  CLOSURE 

PERCEPTUAL  SPEED  CHOICE  REACTION  TIME 
REACTION  TIME 

Reasoning 

INDUCTIVE  REASONING 
CATEGORY  FLEXIBILITY 
DEDUCTIVE  REASONING 
INFORMATION  ORDERING 
MATHEMATICAL  REASONING 
NUMBER  FACILITY 


Cognition 


Visual 

NEAR  VISION  VISUAL  COLOR  DISCRIMINATION 
FAR  VISION  PERIPHERAL  VISION 

NIGHT  VISION  DEPTH  PERCEPTION 
GLARE  SENSITIVITY 

Auditory 

GENERAL  HEARING 
AUDITORY  ATTENTION 
SOUND  LOCALIZATION 


Motor 


Fine  Motor 

CONTROL  PRECISION 
RATE  CONTROL 
WRIST-FINGER  SPEED 
FINGER  DEXTERITY 
MANUAL  DEXTERITY 
ARM-HAND  STEADINESS 
MULTI-LIMB  COORDINATION 

Gross  Motor 

EXTENT  FLEXIBILITY 
DYNAMIC  FLEXIBILITY 
SPEED  OF  LIMB  MOVEMENT 
GROSS  BODY  EQUILIBRIUM 
GROSS  BODY  COORDINATION 
STATIC  STRENGTH 
DYNAMIC  STRENGTH 
TRUNK  STRENGTH 
STAMINA 


Fleishman,  E.  A.  &  Quaintance,  M.  K.  (1984).  Taxonomies  of  human  performance:  The  description  of  human  tasks.  Orlando:  Academic  Press. 

Figure  1.  Job  assessment  software  system  -  50  skills  and  abilities. 
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This  version  of  JASS  was  tailored  for  military  applications  and  was 
developed  using  a  number  of  MOS  test  cases  to  validate  further  the  basic  concepts  in  an 
operational  context  (Knapp  &  Tillman,  1998).  The  JASS  software  was  administered  to  the  soldier 
participants  on  a  laptop  computer  and  required  approximately  an  hour  of  each  soldier’s  time.  Test 
administrators  were  present  to  answer  queries  about  test  or  procedural  matters  related  to  JASS. 

Enhanced  Computer-Aided  Testing  (ECAT) 

The  ECAT  battery  was  developed  jointly  by  the  U.S.  Army  Research 
Institute  and  the  U.S.  Naval  Personnel  Research  and  Development  Center.  It  consists  of  nine 
sub-tests  that  were  designed  to  supplement  the  Armed  Services  Vocational  Aptitude  Battery 
(ASVAB)  now  used  by  the  Department  of  Defense  for  initial  selection  and  training  purposes. 

For  this  effort,  data  were  collected  using  only  two  of  the  nine  sub-tests.  These  particular  tests 
measured  one-  and  two-handed  tracking  performance,  respectively.  The  tests  were  computer 
administered  and  lasted  approximately  20  minutes  each. 

Results 


The  AVO  JASS  data  were  investigated  to  determine  whether  the  requirement  for  high 
levels  of  cognitive  skills  was  pertinent  to  the  flight  issues  discussed.  Figure  2  summarizes  the 
results  and  indicates  that  the  AVO  raters  did  not  consider  their  flight-related  functions  (except 
for  communications)  to  be  overly  demanding  for  any  of  the  skill  clusters.  The  complete  skill 
profiles  are  presented  in  Appendix  B  and  basically  show  the  same  trend.  These  data  are 
supported  by  both  the  accident  reports  reported  next  and  the  feedback  from  SMEs;  the  AVO 
cognitive  skill  level  requirements  do  not  seem  to  be  related  to  flight  issues. 

The  EP  data  were  more  complicated,  and  both  anecdotal  and  empirical  information 
suggest  an  important  relationship  between  the  EP’s  skill  levels  and  safety  (the  data  summary  is 
given  in  Appendix  C).  Figure  3  is  a  bar  graph  plot  of  the  skill  categories  as  a  function  of  skill 
rating.  When  the  EP’s  job  is  compared  with  the  AVO  data,  it  can  be  seen  that  this  job  is  rated  as 
requiring  higher  skill  and  ability  levels  across  all  eight  skill  clusters. 
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Figure  3.  Skill  cluster  ratings  of  external  pilots  (both  high  and  low  experience  levels)  and  rated 
aviators  (both  rotary  and  fixed  wing). 

The  data  are  decomposed  further  into  four  job  categories:  EP  low  experience,  EP  high 
experience,  rotary  wing  aviators,  and  fixed  wing  aviators.  The  main  difference  was  in  the 
reasoning  factor,  with  both  aviator  groups  showing  slightly  increased  importance  attached  to 
reasoning  skills,  compared  to  the  EP  groups.  There  is  evidence  of  relatively  heavy  loadings  on 
conceptual,  vision,  and  psychomotor  components  for  all  groups.  The  EP  low  experience  group 
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seems  to  give  high  ratings  to  the  vision,  audition,  and  psychomotor  skill  clusters.  This  suggests 
that  the  initial  training  may  have  been  particularly  weighted  toward  developing  these  skills. 

The  EC  AT  results  obtained  in  a  previous  study  support  the  particular  importance  of 
psychomotor  skills  during  training.  As  Table  1  indicates,  the  one-  and  two-handed  tracking 
scores  were  nearly  perfect  indicators  of  failure  rate  during  the  EP  training  at  Fort  Huachuca.  Five 
of  the  six  students  who  failed  the  course  had  scores  on  both  the  tracking  tasks  near  the  bottom  of 
the  performance  scores  of  the  sampled  students.  The  EP  designated  “x”  who  also  failed  had  a 
severely  impaired  hand,  making  his  failure  to  complete  the  course  difficult  to  interpret. 

The  data  were  further  analyzed  to  understand  precisely  the  relationship  between  flight 
safety  and  skill  clusters  for  the  four  job  categories.  First,  only  the  task  data  related  to  emergency 
conditions  were  examined  (emergency  landings,  etc.).  Next,  identification  was  made  of  which  of 
the  50  skills  (see  Figure  1  for  the  full  listing)  were  ranked  in  the  top  10  for  each  of  the 
emergency  condition  tasks.  Finally,  determination  was  made  as  to  how  many  of  these  skills 
were  in  each  skill  cluster,  and  these  data  were  plotted  as  a  function  of  what  percentage  of  each 
cluster  was  represented  in  the  top  10.  Based  on  previous  research,  it  was  felt  that  the  importance 
rankings  were  a  better  indicator  of  the  usefulness  of  each  skill  cluster  in  performing  crucial  task 
functions  vice  using  simple  average  skill  values  (Knapp  &  Tillman,  1998).  The  results  are 
plotted  in  Figure  4,  which  shows  a  very  different  relationship  between  experience  level  and  the 
type  of  skills  required  in  emergency  conditions.  The  experienced  EP  used  mostly  conceptual 
skills  in  emergency  situations,  whereas  the  inexperienced  EP  reported  relying  heavily  on  visual 
and  psychomotor  skills  during  these  conditions.  These  findings  are  consistent  with  the  results  of 
the  EC  AT  tracking  tasks  reported  (which  indicated  how  important  the  student’s  perceptual  and 
motor  skills  were  in  passing  the  EP  portion  of  the  UAV  training  regimen).  A  surprising  finding 
was  that  the  aviators  used  speed-loaded  skills  for  emergencies,  whereas  speed-loaded  skills  were 
rated  as  relatively  unimportant  by  both  EP  groups. 

The  UAV  accident  and  air  safety  report  (TSM,  1998)  indicated  that  both  the  Pioneer 
and  the  Hunter  UAVs  historically  had  high  accident  rates  of  an  average  of  one  incident  per 
every  269  and  158  operational  hours,  respectively.  Not  surprisingly,  almost  all  of  the  incidents 
involved  EPs  because  take-off  and  landing  are  the  most  dangerous  parts  of  the  mission  for 
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flight  safety.  However,  since  1996,  the  Hunter  EP  incident  rate  has  fallen  dramatically  to 
1,201  hours  per  incident,  which  compares  favorably  to  the  Predator  (current  Air  Force  UAV) 
rate  of  1,247  hours  per  incident.  One  possibility  for  this  improvement  is  the  maturing  of  the 
Hunter  EP  cadre.  Data  discussed  later  support  this  hypothesis. 

Table  1 

Ranking  of  28  Students  on  the  One-Handed  Tracking  Test  Portions  of  the  ECAT  Inventory 


System  or 
branch  of  service 

EP 

Two-handed 

tracking 

One-handed  tracking 

Hunter-Army 

b 

2729 

2212 

Pioneer-USMC 

r 

4067 

2348 

Hunter-Army 

d 

2829 

2353 

Pioneer-USN 

n 

3537 

2407 

Hunter-USMC 

f 

3738 

2488 

Pioneer-USMC 

t 

3696 

2491 

Hunter-USMC 

e 

3512 

2545 

Pioneer-USMC 

q 

3208 

2605 

Pioneer-USMC 

V 

3271 

2632 

Hunter-USMC 

a 

2852 

2652 

Pioneer-USMC 

z 

3892 

2674 

Hunter-USMC 

g 

3634 

2730 

Pioneer-USN 

u 

3123 

2796 

Pioneer-USMC 

I 

3656 

2800 

Pioneer-USN 

1 

3880 

2837 

Pioneer-USMC 

w 

3953 

2837 

Pioneer-USN 

'  x 
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H96U(EP)-Lo 

■  96U  (EP)  -  High 
□  Aviator-Rotor 

■  Aviator-Fixed 


Figure  4.  Percentage  of  important  skills  used  during  emergencies,  shown  by  job  category. 


Discussion 

Scant  evidence  was  found  for  the  need  of  rated  aviator  skills  for  the  AVO.  The  JASS  list 
of  critical  skills,  accident  data,  and  the  consensus  of  the  SME  deliberations  suggest  that  the  current 
skill  level  of  the  AVO  community  is  sufficient  for  piloting  responsibilities.  The  EP  situation  is 
more  complex.  There  was  a  marked  difference  between  experienced  and  inexperienced  EPs  in  the 
inventory  of  skills  the  two  groups  used  during  emergency  situations.  Apparently,  the  experienced 
EPs  were  able  to  visualize  and  anticipate  problems  before  they  occurred;  an  experienced  UAV 
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operator  described  the  process  as  “getting  ahead  of  the  air  vehicle.”  With  experience,  the  operator 
is  able  to  devote  his  or  her  attentional  resources  to  future  problems  while  attending  to  the  immediate 
perceptual  and  motor  tasks  in  an  automatic  mode.  In  effect,  the  operator  crosses  a  cognitive 
threshold  as  expertise  increases  and  the  problem  domain  becomes  more  cognitive  and  less 
psychomotor  intensive.  This  agrees  with  the  psychological  literature  regarding  both  automatic 
processing  (Shiffrin  &  Schneider,  1977)  and  the  development  of  expertise  (Rasmussen,  1983).  If 
this  interpretation  is  correct,  using  rated  aviators  would  have  little  effect  on  the  accident  rate  during 
landing  and  take-off.  Expertise  tends  to  be  task  specific.  Therefore,  the  specific  motor  skills 
needed  to  control  the  radio-controlled  UAV  would  have  to  be  learned  by  aviators  independently  of 
the  motor  skills  learned  in  flying  an  aircraft.  In  particular,  the  somatic  and  visual  cues  that  pilots 
use  during  aircraft  landings  would  not  be  useful  (and  perhaps  even  counter-productive)  for  the 
different  skill  sets  and  perceptual  viewpoint  necessary  for  radio-controlled  landings.  This  is  not  to 
say  that  there  would  not  be  some  transfer  of  training,  only  that  the  transfer  would  be  transitory,  and 
the  more  cost-effective  solution  would  be  to  develop  expertise  in  the  EP  corps. 

The  improvement  in  the  Hunter  accident  rate  gives  at  least  some  preliminary  assurance  that 
the  EP  performance  record  will  improve  with  the  maturing  of  the  operator  population.  This  does 
not  address  the  question  of  how  to  turn  novices  into  experts.  Fortunately,  innovative  research 
funded  by  the  Israeli  Air  Force  offers  some  promise  in  addressing  this  issue.  Gopher,  Weil,  and 
Bareket  (1994)  developed  a  computer  game  to  help  train  Israeli  Air  Force  cadets  before  flight 
training.  The  computer  game  simulation  was  not  high  fidelity  and  did  not  stress  motor  skills; 
instead,  the  game  emphasized  the  higher  level  conceptual  skills  (such  as  the  ones  identified  in  the 
JASS  for  the  experienced  EPs)  necessary  to  anticipate  and  plan  in  a  combat  aviation  environment. 
The  simulation  group  was  able  to  generalize  these  skill  sets  to  actual  training.  Students  practicing 
the  computer  game  were  twice  as  likely  to  graduate  from  advanced  flight  training  as  the  no-game 
control  groups.  The  Israeli  Air  Force  has  since  adopted  the  computer  game  as  part  of  their  training 
program.  The  UAV  program  would  very  likely  benefit  from  a  similar  computer  training  project. 
The  software  would  be  cost  effective  because  air  vehicle  fidelity  is  not  an  issue;  the  simulation 
would  need  to  emphasize  attentional  and  visualization  skills.  These  skills  could  be  developed  in 
parallel  to  the  psychomotor  and  other  flight  skills  currently  being  developed  in  the  training 
program. 
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A  number  of  related  issues  were  discussed  with  the  SMEs  during  the  consensus  exercise 
held  at  Fort  Huachuca.  The  greater  use  of  speed-loaded  skills  by  the  aviators  at  first  seemed 
counter-intuitive  to  the  SME  group.  However,  further  discussion  suggested  that  the  underlying 
cause  was  related  to  the  demands  of  the  different  aircraft  flown  by  the  two  communities.  The 
controls  and  displays  that  both  fixed  and  rotor  wing  aviators  use  are  extremely  complex, 
especially  compared  to  the  relatively  simple  EP  interface.  Thus,  the  EPs  could  concentrate  on 
future  aircraft  states,  whereas  the  aviators  had  to  respond  to  the  more  complex  interface 
environment  as  well  as  anticipate  future  problems. 

The  question  of  using  rated  aviators  in  either  the  A  VO  or  EP  positions  was  specifically 
addressed  by  the  group  after  the  data  were  presented.  The  group  consensus  was  that  UAV 
operators  do  not  need  to  be  rated  aviators  for  Army  applications.  In  particular,  neither  the  Air 
Force  nor  the  Navy  representatives  believed  that  the  EP  or  AVO  should  necessarily  be  aviator 
rated.  The  Navy’s  solution  was  to  have  the  equivalent  of  the  mission  commander  be  aviator 
rated  when  possible.  This  solution  had  the  advantage  of  freeing  the  AVO  and  EP  to  concentrate 
on  UAV-related  issues,  while  the  mission  commander  handled  the  mission  planning  and  air 
space  coordination,  giving  the  crew  the  benefit  of  his  or  her  aviation  expertise  in  a  supervisory 
role.  The  Air  Force  representative  pointed  out  that  the  Predator  (a  current  Air  Force  UAV)  was  a 
different  air  vehicle  than  those  employed  by  the  Army.  The  Predator  was  designed  to  be  flown 
like  a  standard  aircraft  and  as  such,  the  transfer  of  skills  from  the  aviator  to  the  UAV  community 
was  a  natural  solution.  According  to  this  representative,  no  firm  decision  had  been  made 
concerning  the  use  of  rated  aviators  for  future  Air  Force  UAVs  such  as  Dark  Star. 

In  summary,  there  was  no  evidence  that  would  lead  to  the  conclusion  that  either  the  AVO 
or  the  EP  should  be  rated  aviators.  In  particular,  the  EPs’  landing  and  take-off  functions  require 
motor  and  cognitive  skills  that  are  unique  to  their  mission  profiles  and  job  environment.  However, 
the  greater  use  of  cognitive  skills  by  the  experienced  EPs  suggests  that  greater  emphasis  should  be 
placed  on  developing  these  skills  during  training.  The  use  of  computer  games  was  offered  as  an 
innovative  and  cost-effective  solution  to  accomplish  this  end.  Finally,  the  utility  of  having  military 
aviators  or  personnel  with  equivalent  experience  as  part  of  the  decision  chain  for  UAV  crews 
seems  to  be  both  a  cost-efficient  and  a  tactically  effective  method  to  introduce  aviator  skill  sets 
into  the  UAV  program. 


13 


IMAGERY  AND  INTELLIGENCE  SPECIALISTS  AS  COMPONENTS  OF  THE  UAV  CREW 

As  mentioned,  two  96U  operators  reside  in  the  GCS  during  a  typical  mission.  The  MPO 
works  with  the  AVO  to  search  the  target  area  and  make  preliminary  recognition  and  detection 
decisions  regarding  potential  targets  in  the  locations  designated  by  the  intelligence  staff  as 
named  areas  of  interest  (NAI).  However,  MPOs  are  not  imagery  or  intelligence  analysts,  and 
their  reporting  requirements  in  this  regard  are  minimal.  In  light  of  the  specialized  skills  of  the 
UAV  crews,  the  possibility  of  adding  operators  from  MOSs  with  skills  and  abilities  that 
complement  the  MPOs’  skill  set  was  the  focus  of  this  portion  of  the  study.  The  two  MOSs 
investigated  were  the  96B,  Intelligence  Analyst,  and  the  96D,  Imagery  Analyst. 

Method 

Procedure 

The  JASS  computer-based  job  assessment  system  was  used  as  in  the  rated  aviator 
section  work.  Data  analysis  proved  to  be  fairly  complicated  because  96B  and  96D  MOSs  have 
distinctly  different  task  structures  and  would  therefore  bring  different  skill  sets  to  the  96U  crew. 
In  order  to  assess  the  commonalities  as  well  as  the  differences  among  the  three  jobs,  separate 
task  structures  had  to  be  derived  for  each  of  the  MOS  positions.  From  the  task  lists,  it  was  then 
possible  to  derive  an  overall  ranking  of  the  importance  of  the  JASS  skill  sets  for  each  task 
structure. 


The  actual  comparison  was  done  in  three  steps:  (a)  the  top  20  JASS  skills  (see 
Figure  1)  for  each  of  the  16  tasks  that  the  MPO  performed  were  rank  ordered;  (b)  the  top  20 
skills  for  both  the  96B  and  96D  distinct  skill  sets  were  ranked  separately;  and  (c)  the  resulting 
ranks  of  the  JASS  skills  from  the  96B  and  the  96D  were  compared  to  the  ranks  of  the  JASS  skills 
for  the  96U  operators  for  each  of  the  16  tasks  evaluated  in  the  initial  step.  Kendall’s  rank  order 
correlation  test  was  used  to  evaluate  rank  concordance. 

Participants 

The  comparison  was  made  for  the  tasks  to  which  the  21  96Us  responded  on  the 
JASS  inventory.  Scores  from  nine  96B  analysts  and  eight  96D  imagery  specialists  were 
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collected  on  the  JASS  in  order  to  compare  the  skill  sets  of  these  two  MOSs  to  those  of  the  UAV 
GCS  operators.  All  soldiers  were  stationed  at  Fort  Huachuca. 

Results  and  Discussion 

Table  2  matches  the  UAV  crew  task  duties  to  the  skill  rankings  for  the  96B  and  96D 
operators.  Kendall’s  rank  order  correlation  test  was  used  to  assess  the  commonalities  among  the 
JASS  results.  The  columns  in  Table  2  labeled  “MOS”  indicate  the  degree  of  correlation  between 
the  96D  and  96B  skill  rankings  and  the  rankings  on  each  of  the  duties  listed  in  the  first  column. 
The  96D  skill  rankings  were  significantly  correlated  to  two  of  the  UAV  crew  duties  (p  <  .05).  In 
contrast,  the  96B  showed  a  significant  Kendall  rank  correlation  to  14  of  the  16  duties  the  UAV 
crew  engaged  in  during  their  missions  (again,  p  <  .05).  Interpretation  of  the  data  was  that  the  96D 
was  a  possible  candidate  to  complement  the  skill  profiles  of  the  UAV  crews  because  of  the 
difference  in  the  skill  sets  used  by  these  two  MOS  groups.  In  terms  of  information  theory,  the 
lack  of  redundancy  between  the  two  MOSs  implies  a  higher  information  transmission  rate.  The 
authors’  interpretation  was  given  credibility  by  the  SME  discussions  that  indicated  the  importance 
of  enhancing  the  imagery  interpretation  skills  of  the  MPO  in  particular.  It  was  felt,  especially  by 
the  96U  operator  participants,  that  the  96D  skills  would  be  a  very  useful  addition  to  the  UAV 
crew.  This  does  not  imply  that  MPO  requires  the  in-depth  imagery  understanding  of  the  96D;  the 
96D  skills  could  be  employed  remotely  at  the  brigade  or  division  tactical  operations  center 
(TOC).  For  many  or  perhaps  even  most  missions,  the  detection  and  recognition  reporting  skills  of 
the  MPO  would  suffice  to  meet  the  commander’s  goals.  The  96D  skills  would  be  necessary  for 
particularly  difficult  interpretations  or  specialized  missions  when  in-depth  target  analyses  are 
required.  Another  possibility  would  be  to  incorporate  the  96D  skills  into  the  mission  command 
module  by  enhancing  the  skill  set  of  the  data  exploitation  operator  (DEO)  with  additional  imagery 
training.  The  DEO  resides  in  the  command  module  and  performs  the  function  of  a  senior  analyst 
but  is  not  currently  required  to  have  96D  training.  In  summary,  the  principal  conclusion  is  that 
additional  imagery  support  using  96D  specialists  should  enhance  the  overall  operational 
versatility  and  capabilities  of  the  UAV  crews.  On  the  other  hand,  the  role  of  the  96B  as  now 
configured  seems  to  be  a  satisfactory  adjunct  to  the  UAV  crews’  intelligence-gathering  function. 
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Table  2 


Statistical  Comparison  of  Skill  Commonalities  Using 
the  Kendall  Rank  Order  Correlation  Test 


96U  duties 

96D  highest 
overall  skills 

96B  highest 
overall  skills 

Create  air  vehicle  mission  plan  on  display 

** 

Perform  air  reconnaissance 

** 

Perform  air  vehicle  navigation 

** 

Prepare  air  vehicle  mission  plan 

** 

Detect  targets  of  military  significance 

** 

Identify  target  type  and  number 

** 

Operate  remote  video  terminal 

** 

Perform  mission  payload  terminal 

** 

Recognize  targets;  place  in  context 

Transfer  control  of  air  vehicle 

** 

Prepare  intelligence  reports 

** 

Disseminate  mission  results 

** 

** 

Coordinate  airspace  requirements 

** 

Coordinate  with  higher  headquarters 

** 

Coordinate  with  support  and  external  elements 
Conduct  launch  and  recovery  operations 

** 

** 

**Significance  level:  p  <  .05 


AUTOMATION  AND  WORKLOAD  MODELING  FOR  FUTURE  UAV  PLATFORMS 

An  important  consideration  in  designing  the  future  crew  interfaces  is  the  degree  and  type  of 
automation  required  in  future  UAV  applications.  The  UAV  operator  has  to  perform  multiple 
functions,  often  simultaneously  during  a  typical  mission  profile  (Barnes  &  Matz,  1998).  In  order 
to  understand  automation  requirements  in  this  environment,  the  MicroSaint  modeling  environment 
was  used  to  investigate  the  workload  for  one  potential  future  UAV  platform,  the  Outrider.  The 
Outrider  was  a  good  candidate  to  investigate  incipient  crew  workload  issues  (i.e.,  high  workload 
may  suggest  a  need  to  automate  tasks)  because  the  Outrider  was  in  the  process  of  completing  an 
advanced  combat  technology  development  (ACTD)  during  these  data  collection  efforts.  MicroSaint 
was  chosen  because  it  is  a  relatively  mature  instrument  and  has  been  used  successfully  in  a  number 
of  human  engineering  applications.  (A  detailed  description  of  MicroSaint  is  given  in  Appendix  D.) 
However,  the  general  findings  of  this  report  should  generalize  to  a  larger  class  of  PC  workload 

.  TM  . 

modeling  environments;  in  particular,  the  underlying  workload  model  residing  in  MicroSaint  is 
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shared  with  other  test  instruments  such  as  the  Improved  Performance  Research  Integration  Tool 
(IMPRINT). 

Method 

Procedure 

First,  a  model  of  the  Hunter  UAV  system  was  developed  by  using  MicroSaint™ 
and  a  database  that  contained  most  of  the  GCS  operator  tasks  and  functions  related  to  the  Hunter 
system,  which  range  from  setting  up  the  equipment,  route  planing,  internal  flight  procedures,  and 
intelligence  gathering  to  actually  landing  the  UAV.  The  Hunter  model  was  based  mainly  on  a 
Hardman  III  workload  task  analysis1  done  for  the  Joint  Tactical  UAV  Program  Office  as  part  of  a 
previous  project.  In  addition  to  task  time  data  and  the  task  sequence  logic,  the  database 
contained  the  visual,  auditory,  cognitive,  and  psychomotor  workload  values  for  each  task.  This 
model  served  as  a  foundation  for  the  design  of  the  Outrider  model. 

The  Hunter  model  was  then  modified  according  to  information  from  SMEs  and 
data  collected  during  an  observation  of  the  Outrider  training  simulator.  The  scenario  chosen  to  be 
used  in  the  model  included  four  stationary  targets,  no  malfunctions,  and  no  in-flight  modifications. 
After  the  model  was  executed,  two  sets  of  data  were  produced:  the  workload  values  for  each 
operator  throughout  the  scenario  and  the  number  of  steps  required  to  perform  each  task. 

Participants 

The  number  of  SMEs  available  to  assist  in  building  the  Outrider  model  was  small; 
however,  the  scarcity  of  the  subject  pool  was  mitigated  by  drawing  upon  an  existing  network 
model  of  the  Hunter  UAV,  which  had  been  validated  during  a  number  of  simulation  exercises 
(Barnes  &  Matz,  1998).  The  first  iteration  of  knowledge  elicitation  was  done  at  Fort  Huachuca 
with  two  experienced  UAV  operators  who  were  familiar  with  the  Outrider  and  a  human  factors 
specialist  familiar  with  the  previous  workload  model  developed  for  the  Hunter  in  the  1993-1995 
time  frame.  The  next  iteration  was  completed  at  Fort  Hood  using  two  96U  soldiers  who  had 


‘Test  battery  developed  by  ARI 


17 


been  trained  the  month  before  in  the  Outrider  training  simulator  in  Hondo,  Texas.  The  last 
iteration  took  place  in  Hondo  with  two  SMEs  whose  job  was  to  develop  lesson  plans  for  the 
training  simulator  and  to  teach  96U  operators  to  use  the  Outrider  simulator.  Both  operators  had 
been  flight-qualified  Hunter  operators  before  being  employed  in  their  current  positions. 

Workload  Scales 

The  visual,  auditory,  cognitive,  and  perceptual  (VACP)  workload  theory 
implemented  in  this  work  is  discussed  in  detail  in  an  Army  Research  Institute  report  (McCracken 
&  Aldrich,  1984). 

Workload  theory  is  based  upon  the  idea  that  every  task  a  human  performs  requires 
some  effort  or  work.  Usually,  a  task  is  composed  of  several  different  types  of  work,  such  as 
visual  or  cognitive.  For  example,  consider  a  task  such  as  steering  a  car.  This  task  will  have 
some  visual  work  (watch  where  you  are  going),  some  cognitive  work  (decide  if  you  are  turning 
enough),  and  some  psychomotor  work  (rotate  the  steering  wheel).  The  workload  theory 
implemented  in  this  effort  assigns  values  representing  the  amount  of  effort  that  must  be 
expended  in  each  channel  in  order  to  perform  the  task.  Table  3  scales  are  taken  directly  from 
Bierbaum,  Szabo,  and  Aldrich  (1989). 

This  theory  also  hypothesizes  that  when  two  tasks  are  performed  at  once,  the 
workload  levels  are  additive  within  channels,  across  tasks.  For  example,  if  two  tasks  are  being 
done  at  once,  one  with  a  psychomotor  load  of  2.6  and  one  with  a  psychomotor  load  of  4.6,  then  a 
psychomotor  score  of  7.2  (2.6  +  4.6)  would  be  recorded  for  the  time  that  the  two  tasks  were 
being  performed  together. 

Results 

Four  different  categories  of  data  were  collected  to  help  determine  which  tasks  should  be 
candidates  for  automation.  These  categories  were  based  on  the  model  output  and  data  taken 
from  interviews  with  the  SMEs.  Besides  the  two  model-based  data  sources,  the  SMEs  provided 
a  list  of  tasks  that  were  critical  to  the  mission,  and  they  indicated  which  additional  tasks  they 
would  like  to  see  automated. 
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Table  3 


Workload  Scale  Values 


Scale 

Scale  value 

Descriptor 

Auditory  scale 

0.0 

No  auditory  activity 

1.0 

Detect  or  register  sound  (detect  occurrence  of  sound) 

2.0 

Orient  to  sound  (general  orientation  or  attention) 

4.2 

Orient  to  sound  (selective  orientation  or  attention) 

Auditory  workload 

4.3 

Verify  auditory  feedback  (detect  occurrence  of  anticipated  sound) 

4.9 

Interpret  semantic  content  (speech) 

6.6 

Discriminate  sound  characteristics  (detect  auditory  differences) 

7.0 

Interpret  sound  patterns  (purse  rates,  etc.) 

Cognitive  scale 

0.0 

No  cognitive  activity 

1.0 

Automatic  (simple  association) 

1.2 

Alternative  selection 

Cognitive  workload 

3.7 

Sign  or  signal  recognition 

4.6 

Evaluation  or  Judgment  (consider  single  aspect) 

5.3 

Encoding  or  decoding,  recall 

6.8 

Evaluation  or  judgment  (consider  several  aspects) 

7.0 

Estimation,  calculation,  conversion 

Psychomotor  scale 

0.0 

No  psychomotor  activity 

1.0 

Speech 

2.2 

Discrete  actuation  (button,  toggle,  trigger) 

2.6 

Continuous  adjustive  (flight  control,  sensor  control) 

4.6 

Manipulative 

Psychomotor  workload 

5.8 

Discrete  adjustive  (rotary,  vertical  thumb  wheel,  lever  position) 

6.5 

Symbolic  production  (writing) 

7.0 

Serial  discrete  manipulation  (keyboard  entries) 

Visual  scale 

0.0 

No  visual  activity 

1.0 

Visually  register  or  detect  (detect  occurrence  of  image) 

3.7 

Visually  discriminate  (detect  visual  differences) 

Visual  workload 

4.0 

Visually  inspect  or  check  (discrete  inspection  or  static  condition) 

5.0 

Visually  locate  or  align  (selective  orientation) 

5.4 

Visually  track  or  follow  (maintain  orientation) 

5.9 

Visually  read  (symbol) 

7.0 

Visually  scan,  search,  or  monitor  (continuous  or  serial  inspection, 
multiple  conditions) 

Automation  is  generally  suggested  for  tasks  (a)  that  have  high  workload,  (b)  that  require 
multiple  operator  actions,  (c)  that  are  mission  critical  or  life  threatening,  and  (d)  the  operator 
feels  are  auxiliary  or  bookkeeping,  which  could  be  automated  easily.  The  four  categories  of  data 
(workload,  steps  per  task,  critical  tasks,  and  operator  suggestions)  were  analyzed  to  identify 
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which  tasks  might  be  automated.  Tasks  that  appear  in  multiple  categories  were  then  reviewed 
for  a  final  recommendation  about  the  requirement  for  automation. 

Workload 

Each  task  within  the  Outrider  model  has  corresponding  visual,  auditory,  cognitive, 
and  psychomotor  workload  values.  Tasks  that  have  workload  values  of  5.2  or  higher  in  at  least 
two  of  the  workload  components  were  viewed  as  high  workload  tasks  and  are  listed  next: 

•  Enter  way  points 

•  Verify  system  settings 

•  Monitor  video,  telemetry,  and  systems 

•  Check  AV  and  navigation  systems 

•  Enter  way  points  and  prepare  flight  plan 

Steps  Per  Task 

Each  task  within  the  Outrider  model  is  performed  in  one  or  more  steps.  The  tasks 
with  three  or  more  steps  involved  are 

•  Set  up  equipment 

•  Perform  off-line  mission  planning 

•  Enter  way  points 

•  Analyze  and  modify  mission  planning 

•  Verify  system  settings 

•  Perform  engine  start  procedures 

•  Perform  verifications 

•  Monitor  video,  telemetry,  and  systems 

•  Check  AV  and  navigation  systems 

•  Monitor  flight  and  search  parameters 

•  Enter  way  points  and  prepare  flight  plan 

•  Monitor  landing 

•  Modify  landing 
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•  Perform  checks  after  landing 


Critical  Tasks 

The  functions  that  must  be  performed  in  order  for  the  mission  to  be  completed  are 


Set  up  equipment 
Set  up  map  system 
Create  mission  plan 
Preflight 
Verify  indicators 
Start  engine 

Perform  take-off  procedures 
Fly  to  way  points 
Perform  area  search 
Recover  AY 


Tasks  that  operators  suggested  are 

•  Analyze  and  modify  mission  plan 

•  Perform  pre-flight  functions 


The  tasks  that  appear  in  two  or  three  of  the  categories  are  listed  next.  No  tasks  appeared 
in  all  four  categories.  Tasks  from  the  function  “set  up  equipment”  were  removed  because  they 
cannot  be  automated.  Tasks  from  the  function  “perform  off-line  mission  planning”  were  also 
removed  because  it  is  a  non-critical  function  that  is  usually  performed  only  during  training  and 
because  the  UAV  operators  already  perform  mission  planning  “on  line”. 

•  Enter  way  points 

•  Analyze  and  modify  mission  plan 

•  Perform  pre-flight  procedures 

•  Verify  system  settings 

•  Perform  engine  start  procedures 
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•  Monitor  video,  telemetry,  and  systems 

•  Check  AV  and  navigation  systems 

•  Enter  way  points  and  prepare  flight  plan 

•  Monitor  landing 

•  Modify  landing 

•  Perform  checks  after  landing 

Discussion 

The  results  indicate  that  the  candidates  for  automation  include  pre-  and  post-flight 
procedures  and  checks,  verification  of  system  settings,  and  computer  checks  for  the  mission 
plans.  This  corresponds  with  the  suggestions  provided  by  the  SMEs  who  stated  that  although  the 
Outrider  system  does  provide  some  error  messages,  it  does  not  check  to  see  if  the  mission  plan  or 
system  settings  are  within  range  or  engineering  limits.  In  addition,  the  results  indicate  that 
monitoring  is  another  task  that  could  be  automated.  However,  monitoring  the  aircraft  is  one 
reason  why  human  operators  are  involved  in  the  “loop”.  Still,  this  task  can  be  partially 
automated  (e.g.,  warnings  or  voice  commands  can  be  given  by  the  system  when  certain 
parameters  are  no  longer  within  specified  values).  In  particular,  when  system  safety  is  involved, 
having  both  the  human  and  the  system  computer  monitor  for  possible  safety  issues  is  essential. 
The  task  “modify  landing”  addresses  the  issue  of  unsafe  landings  and  would  entail  extensive 
analyses  to  determine  the  optimal  mixture  of  human  and  computer  control  during  dangerous 
landing  situations.  In  general,  the  operators  were  not  asking  for  fully  automated  systems; 
instead,  they  preferred  the  decision  making  to  remain  with  the  operator  and  the  workload 
reduction  to  be  accomplished  by  making  the  computer  interface  faster  and  more  efficient  as  well 
as  having  the  computer  become  another  set  of  “eyes”  to  check  for  safety  problems. 

It  is  also  important  to  determine  how  operators  react  when  the  system  behaves 
unexpectedly  and  which  corrective  tasks  should  be  automated  or  computer  aided.  Areas  for 
future  work  include  expanding  the  model  to  simulate  more  scenarios,  such  as  instances  of 
dynamic  targets  and  system  malfunctions,  and  to  collect  human  performance  to  extend  the 
model’s  capabilities  to  predict  mission  and  task  outcomes.  Further  investigation  is  also  needed 
to  examine  the  human  cognitive  profile  related  to  search  tasks  and  to  assess  the  utility  of 
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automated  search  and  target  detection  algorithms.  Finally,  the  model  should  be  improved  so  that 
it  is  possible  to  examine  how  fatigue  and  possibly  stress  factors  affect  operator  performance  and 
overall  mission  safety  in  future  UAV  operational  tempos. 


GENERAL  DISCUSSION 

The  use  of  a  variety  of  human  engineering  tools  has  helped  in  our  understanding  of  future 
crew  environments.  Most  of  the  results  were  generic  and  can  be  used  to  help  guide  the  design 
process  for  any  UAV  configurations  involved  in  tactical  Army  missions.  For  example,  the 
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MicroSaint  model  generated  a  number  of  hypothetical  task  structures  for  possible  automation, 
which  should  generalize  to  most  future  tactical  UAV  environments.  These  tasks  can  be  narrowed 
further  by  design  considerations,  and  realistic  soldier-in-the-loop  simulation  experiments  can  then 
be  designed  to  focus  on  a  small  set  of  pre-selected  tasks.  The  results  of  the  JASS  study  for  the 
rated  pilots  were  supplemented  by  performance  data  from  both  training  and  accident  data  that 
indicated  the  ability  of  these  techniques  to  combine  easily  with  empirical  methods.  Another 
feature  of  the  analyses  was  the  reliance  on  the  SME  team  for  interpretation.  This  is  probably 
inevitable  in  a  developing  system  because  no  one  person  could  possibly  understand  the  tactical, 
programmatic,  and  engineering  issues  of  a  system  that  is  yet  to  be  developed.  The  backgrounds  of 
the  SMEs  involved  were  broad  enough  to  cover  many  of  these  facets,  thus  laying  a  firm  foundation 
for  further  analyses.  Also,  the  combination  of  modeling  techniques  and  expert  input  helped  to 
curtail  the  shortcomings  of  both  approaches  by  constraining  the  experts’  tendency  to  tell  “war 
stories”  and  by  giving  the  results  of  the  modeling  efforts  face  validity  and  an  operational  context. 

The  preliminary  suggestions  for  the  UAV  program,  which  were  derived  across  the  three 
sets  of  analyses,  are 

1.  It  is  not  cost  efficient  to  require  flight  certification  for  either  the  A  VO  or  EP  operator 
positions. 

2.  Computerized  training  (especially  embedded  training)  should  be  an  effective  means 
for  developing  operator  flight  skills.  These  efforts  should  concentrate  on  the  cognitive 
components  of  the  flight  tasks. 
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3.  Aviator-rated  personnel  (or  personnel  with  equivalent  expertise)  should  be  involved  in 
the  decision  chain  to  aid  the  UAV  crew  in  mission  planning,  air  space  coordination,  and  general 
liaison  with  the  other  services. 

4.  Imagery  interpretation  skills  drawn  from  the  96D  training  program  would  be  a  useful 
addition  to  the  UAV  targeting  and  reporting  process.  These  skills  do  not  have  to  be  present  in 
UAV  ground  control  stations. 

5.  Automation  requirements  for  the  UAV  operator  should  focus  on  computer  assistance 
(e.g.,  quickly  change  way  points)  and  system  monitoring  rather  than  on  acquiring  fully 
automated  sub-systems.  (Note.  The  utility  of  automated  landing  and  take-off  was  not  addressed 
in  this  study  because  the  status  of  this  feature  on  the  Outrider  was  not  clear  at  the  time  the 
workload  data  were  collected.) 

6.  Future  modeling  efforts  should  include  human  performance  (particularly  in  the  search 
domain)  and  fatigue  and  stress  data  to  predict  mission  performance  during  future  UAV 
operational  tempos  more  effectively. 

The  basic  premise  of  this  effort  is  that  by  using  a  variety  of  human  engineering  methods, 
a  set  of  tools  and  methods  could  be  created,  which  will  mutually  reinforce  each  other.  The 
authors  deliberately  chose  to  investigate  methods  that  were  both  cost  and  time  efficient,  thus 
avoiding  methods  that  required  large-scale  simulations  or  field  exercises.  MicroSaint  was 
chosen  in  part  because  it  is  available  on  personal  computers  and  its  software  is  relatively 
inexpensive  and  easy  to  use.  The  overall  goal  is  to  improve  the  human  engineering  design 
process  by  introducing  methods  (particularly  computerized  ones)  that  encourage  early  human 
system  integration  (HSI)  analysis  before  the  traditional  materiel  acquisition  process  begins.  Too 
often,  especially  early  in  the  acquisition  process,  the  amount  of  HSI  analysis  is  determined  by 
cost  and  timeliness  considerations.  Tools  such  as  JASS,  ECAT,  IMPRINT,  and  MicroSaint  are 
being  continually  refined  and  validated  to  be  more  efficient  and  scientifically  valid.  The  strategy 
adapted  here  is  to  combine  these  methods  for  a  synergistic  approach  that  can  be  used  to 
investigate  a  complex  and  changing  HSI  environment  early  in  the  design  process. 
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JASS  AVERAGES  FOR  AVO  POSITION  ACROSS  DUTIES 
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JASS  AVERAGES  FOR  AVO  POSITION  ACROSS  DUTIES 
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Average  score  within  each  skill  cluster  across  16  duties  for  the  A  VO 


Communication 

AVO 

Oral  Comprehension 

4.41 

Written  Comprehension 

3.72 

Oral  Expression 

3.96 

Written  Expression 

2.61 

AVERAGE 

3.68 

Conceptual 

AVO 

Memorization 

3.24 

Problem  Sensitivity 

Originality 

1.02 

Fluency  of  Ideas 

0.89 

Flexibility 

1.78 

Selective  Attention 

2.53 

Spatial  Orientation 

3.00 

Visualization 

1.18 

AVERAGE 

2.13 

Reasoning 

AVO 

Inductive  Reasoning 

1.23 

Category  Flexibility 

0.79 

Deductive  Reasoning 

3.13 

Information  Ordering 

2.75 

Mathematical  Reasoning 

1.14 

Number  Facility 

0.62 

AVERAGE 

1.61 

Speed-loaded 

AVO 

Time  Sharing 

1.84 

Speed  of  Closure 

1.14 

Perceptual  Speed  and 
Accuracy 

2.33 

Reaction  Time 

0.53 

Choice  Reaction  Time 

1.62 

AVERAGE 

1.49 
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Vision 

AVO 

Near  Vision 

Far  Vision 

1.64 

Night  Vision 

1.66 

Visual  Color  Discrimination 

1.13 

Peripheral  Vision 

1.23 

Depth  Perception 

Glare  Sensitivity 

1.02 

AVERAGE 

1.20 

Audition 

AVO 

General  Hearing 

0.16 

Auditory  Attention 

Sound  Localization 

AVERAGE 

0.21 

Psychomotor 

AVO 

Control  Precision 

1.65 

Rate  Control 

0.97 

Wrist-Finger  Speed 

0.29 

Finger  Dexterity 

1.58 

Manual  Dexterity 

0.71 

Arm-hand  Steadiness 

1.31 

Multi-Limb  Coordination 

0.72 

AVERAGE 

1.03 

Gross  Motor 

AVO 

Extent  Flexibility 

Dynamic  Flexibility 

0.00 

Speed  of  Limb  Movement 

1.06 

Gross  Body  Equilibrium 

Gross  Body  Coordination 

Static  Strength 

0.00 

Explosive  Strength 

0.00 

Dynamic  Strength 

0.00 

Trunk  Strength 

0.00 

Stamina 

0.00 

AVERAGE 

0.12 
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APPENDIX  C 

JASS  DATA  AVERAGES  FOR  EP  (LOW  AND  HIGH  EXPERIENCE)  AND  FIXED 
AND  ROTARY  WING  AVIATORS’  POSITION  ACROSS  DUTIES 
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JASS  DATA  AVERAGES  FOR  EP  (LOW  AND  HIGH  EXPERIENCE)  AND  FIXED 
AND  ROTARY  WING  AVIATORS’  POSITION  ACROSS  DUTIES 
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Skill  Cluster 


Average  score  within  each  skill  cluster  across  nine  duties 


Communication 


Oral  Comprehension 


Written  Comprehension 


Oral  Expression 


Written  Expression 


AVERAGE 


Conceptual 


Memorization 


Problem  Sensitivity 


Originality _ 


Fluency  of  Ideas 


Flexibility _ 


Selective  Attention 
Spatial  Orientation 


Visualization 


AVERAGE 


EP 

inexperienced 

(low) 

5.69 


2.83 

5.57 


1.59 


3.92 


_ Group 

EP  experienced 
(high) 


Rotary 


EP 

inexperienced 
(low) _ 


5.97 

5.72 


Grou 


EP  experienced 
(high) 


Rotary 


Fixed 


Fixed 


Reasoning 


Inductive  Reasoning 


Category  Flexibility 


Deductive  Reasoning 


Information  Ordering 


Mathematical  Reasoning 


Number  Facility _ 

AVERAGE 


EP 

inexperienced 

(low) 


Group 


EP  experienced 
(high) 


Rotary 


Fixed 


Speed-loaded 


Time  Sharing _ 

Speed  of  Closure 
Perceptual  Speed  and 

Accuracy _ 

Reaction  Time _ 

Choice  Reaction  Time 
AVERAGE 


Vision 


Near  Vision 

Far  Vision _ 

Night  Vision 
Visual  Color 
Discrimination 
Peripheral  Vision 
Depth  Perception 
Glare  Sensitivity 
AVERAGE 


Audition 


General  Hearing 
Auditory  Attention 
Sound  Localization 
AVERAGE 


Psychomotor 


Control  Precision _ 

Rate  Control _ 

Wrist-Finger  Speed 

Finger  Dexterity _ 

Manual  Dexterity _ 

Arm-hand  Steadiness 
Multi-Limb  Coordination 
|  AVERAGE 


EP 

inexperienced 


_ Group 

EP  experienced 
(high) 


Rotary 


Fixed 


(low) 

4.38 

4.26 

4.08 

5.20 

5.79 

5.17 

5.13 

4.86 


4.64 

3.11 

2.18 

4.59 

3.72 

4.36 

2.20 

3.54 


5.32 

2.77 
2.95 
3.18 
4.54 
3.90 

4.77 
3.92 


4.51 

3.93 

2.35 

0.96 

3.32 

2.76 

4.07 

3.13 


Group 

Gross  Motor 

EP 

inexperienced 

(low) 

EP  experienced 
(high) 

Rotary 

Fixed 

Extent  Flexibility 

1.07 

1.04 

1.15 

1.84 

Dynamic  Flexibility 

1.07 

■QBi 

0.14 

1.52 

Speed  of  Limb  Movement 

0.15 

1 

2.97 

2.30 

Gross  Body  Equilibrium 

2.16 

1.35 

1.91 

Gross  Body  Coordination 

0.00 

0.73 

Static  Strength 

0.00 

0.00 

0.87 

0.81 

Explosive  Strength 

HHEEEHH1 

0.00 

0.00 

0.85 

Dynamic  Strength 

0.00 

0.00 

0.47 

Trunk  Strength 

■ 

0.00 

0.12 

Stamina 

| 

0.00 

0.24 

AVERAGE 

0.45 

0.37 

0.52 

1.08 
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DISCRETE  EVENT  SIMULATION  USING  MICROSAINT™ 
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DISCRETE  EVENT  SIMULATION  USING  MICROSAINT™ 

Discrete  event  simulations  (DES)  use  a  computer  model  to  describe  a  process  that  can  be 
expressed  as  a  sequence  of  events,  each  with  a  distinct  beginning  and  end.  Events  can  be  any 
part  of  the  process,  such  as  scheduled  activities  or  tasks  that  represent  the  flow  of  the  process. 

The  tasks  are  displayed  schematically  on  a  diagram  called  the  task  network  diagram,  which  is  the 
basis  of  the  model. 

MicroSaint™  is  a  simulation  software  package  for  constructing  models  that  simulate  real- 
life  processes.  In  this  section,  the  basic  DES  components  that  comprise  the  MicroSaint™ 
software  tool  are  described.  Models  can  be  relatively  simple  or  complex.  A  simple,  functional 
model  can  be  built  just  by  creating  a  network  diagram  and  entering  task  timing  information  for 
each  task  in  the  network.  More  complex  models  can  be  built,  which  include  dynamically 
changing  variables,  probabilistic  and  tactical  branching  logic,  conditional  task  execution,  and 
extensive  model  data  collection — all  of  which  can  be  specified  by  choosing  menu  commands  or 
providing  expressions  for  MicroSaint™  to  execute  during  specific  circumstances. 

Whether  the  model  is  simple  or  complex,  the  process  of  executing  a  MicroSaint™  model 
and  generating  statistics  and  graphs  from  the  collected  data  is  mostly  automatic.  The  software 
uses  random  numbers  to  generate  specific  task  times  from  a  pre-established  distribution  and 
routing  choices  specific  to  the  current  execution.  After  the  model  has  been  run,  statistic  charts, 
scatter  plots,  line  or  step  graphs,  bar  charts,  and  frequency  distributions  can  be  used  to  analyze 
the  data  collected  during  model  execution.  In  addition,  the  results  files  can  be  opened  in 
spreadsheets  or  statistical  packages  for  further  analysis. 

This  section  is  designed  to  provide  sufficient  information  about  MicroSaint™  so  that  the 
Outrider  UAV  modeling  presented  in  this  report  can  be  understood.  This  is  not  meant  to  provide  a 
complete  understanding  of  how  MicroSaint™  can  be  used  for  modeling  in  general.  For  questions 
and  a  more  detailed  understanding  of  MicroSaint™,  refer  to  the  MicroSaint™  3.0  manual. 
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User  Interface 


MicroSaint™  uses  a  standard  Windows™-style  graphical  interface.  The  standard  point- 
and-click  method  is  used  to  select  MicroSaint  tools  and  to  define  and  move  objects.  Double¬ 
clicking  an  object  with  the  mouse  opens  a  description  dialog  box  where  information  specific  to 

TM 

the  object  can  be  entered.  Figure  D-l  shows  the  task  network  diagram  window  of  MicroSaint  . 
The  window  contains  a  sample  network  diagram  of  four  nodes  labeled  1  through  4,  with  a 
probabilistic  decision  node  after  Node  2. 


Task  Network  Diagram 

The  task  network  is  a  graphical  representation  of  the  process  that  is  being  modeled. 
Tasks  are  represented  in  a  diagram  that  shows  the  order  of  task  execution  within  the  process.  A 
task  network  diagram  is  composed  of  nodes  representing  tasks  that  are  connected  by  arrows.  A 
rounded  rectangle  or  oval  shape  represents  each  task.  Sub-networks  are  represented  by  a 
rectangle.  The  arrows  between  the  nodes  indicate  the  possible  sequences  in  which  the  tasks  can 
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be  performed.  Figure  D-l  is  an  example  of  a  task  network.  The  “P”  in  the  diamond-shaped  node 
represents  the  type  of  decision  (probabilistic)  that  is  used  to  determine  which  path  is  taken. 

Task  and  network  nodes  are  created  in  MicroSaint™  with  the  task  and  network  tools. 

Users  click  on  the  network  diagram  with  one  of  the  tools  to  place  a  task  or  network  and  then 
continue  clicking  to  place  subsequent  tasks  (or  networks).  The  path  tool  is  used  to  draw  a  path 
from  each  task  or  network  to  any  other  task  or  network  that  can  follow  it,  and  it  indicates  the 
direction  of  task  execution.  MicroSaint™  also  uses  symbolic  animation  during  execution.  When  a 
particular  task  in  the  network  has  been  reached,  the  rounded  rectangle  for  that  task  is  highlighted. 
The  animation  shows  entities  (items,  people,  etc.)  as  they  move  through  the  network.  This  type  of 
animation  is  particularly  useful  in  debugging  a  model  and  when  verifying  a  model  with  SMEs. 

Task  Description 

Tasks  are  the  lowest  level  in  a  model  network  hierarchy  and  are  described  by  specific 
parameters  such  as  task  timing  information,  release  condition,  and  beginning  and  ending  effect, 
which  relate  the  task  to  other  system  activities.  An  example  of  the  task  description  dialog  box  is  ■ 
displayed  in  Figure  D-2.  The  description  is  for  Task  Number  1  (this  number  is  inside  the 
MicroSaint™  software  and  does  not  affect  or  reference  the  process  being  modeled);  a  name  for 
the  task  can  be  entered  into  the  name  field.  Expressions  for  each  of  the  task  parameters  can  be 
entered  in  the  labeled  fields. 

Task  Timing  Information 

Task  timing  information  consists  of  the  mean  time  for  the  task,  the  standard 
deviation,  and  a  type  of  time  distribution.  In  Figure  D-2,  the  task  mean  time  is  10  time  units 
(hours,  minutes,  seconds,  etc.),  the  standard  deviation  is  one  time  unit,  and  the  time  distribution 
is  normal. 


The  mean  time  is  the  average  time  required  to  complete  a  task.  For  example,  if 
the  task  represents  an  activity  such  as  “enter  way  points,”  then  the  mean  time  to  execute  the  task 
is  the  average  time  that  it  takes  an  operator  to  enter  the  way  points.  The  mean  time  is  used  in 
conjunction  with  the  standard  deviation  and  time  distribution  to  determine  the  simulated  task 


51 


execution  time  for  each  execution  of  the  task.  The  standard  deviation  is  used  in  conjunction  with 
the  time  distribution  and  controls  the  spread  of  a  distribution 


Figure  D-2.  Task  description  dialogue  box. 

The  time  distribution  indicates  the  function  used  by  MicroSaint  to  randomly 
generate  execution  times  for  a  task.  The  mean  time  and  standard  deviation  are  used  in 
conjunction  with  the  probability  distribution  to  determine  the  task  execution  time.  In  most  cases, 
the  execution  time  is  not  constant,  but  instead,  the  execution  time  is  variable  within  a  range  of 
values  that  can  be  represented  by  a  probability  distribution.  MicroSaint  supports  more  than  21 
probability  distribution  types,  including  normal,  rectangular,  exponential,  gamma,  Wiebull, 
Poisson,  triangular,  and  others. 

Release  Conditions 

Situations  often  occur  when  a  task  cannot  begin  executing  until  certain  conditions 
are  met.  A  task  can  have  resource  requirements  such  as  availability  of  an  operator  or  other 


constraints  such  as  time  of  day  or  availability  of  part  type  that  controls  when  the  task  can  begin. 
In  MicroSaint™,  the  expression  in  the  “release  condition”  field  can  prevent  a  task  from  executing 
until  certain  conditions  in  the  model  are  met  (e.g.,  the  availability  of  a  resource,  the  completion 
of  another  task).  The  release  condition  expression  can  be  as  simple  as  the  value  1  for  tasks  that 
execute  as  soon  as  the  previous  task  completes,  or  it  may  be  a  complicated  expression  in  which 
several  conditions  are  evaluated.  Entities  moving  through  the  network  cannot  be  released  into  a 
task  for  processing  until  the  release  conditions  for  the  task  are  met. 

Task  Execution  Effects 

An  execution  effect  defines  how  the  task  performance  affects  other  aspects  of  the 
system.  For  instance,  the  current  state  of  the  system  may  change  when  a  task  begins  and  then 
change  again  when  the  task  ends.  These  changes  are  made  using  expressions  in  the  beginning 
and  ending  effects  of  a  task  description.  In  the  example  in  Figure  D-2,  the  expression  in  the 
beginning  effect  of  the  task  reduces  the  number  of  available  operators  by  one.  The  expression  in 
ending  effect  increases  the  number  of  available  operators  by  one. 

Controlling  Process  Logic 

The  arrows  that  are  displayed  between  nodes  define  the  basic  order  in  which  tasks  are 
executed.  Alternatives  are  indicated  when  more  than  one  path  is  displayed,  which  originated 
from  a  single  node.  Task  sequences  can  also  be  affected  by  conditions  outside  the  network 
diagram.  For  example,  a  task  can  be  started  as  a  function  of  time.  A  diamond-shaped  “decision 
node”  automatically  displays  on  the  network  diagram  when  more  than  one  path  follows  a  task. 
These  decision  points  can  be  used  to  represent  real-world  decisions  or  to  control  aspects  of  how 
the  model  works,  which  may  have  little  to  do  with  the  process  being  modeled. 

The  conditions  that  control  the  branching  must  be  entered  as  expressions.  MicroSaint™ 
provides  the  following  decision  types  to  ensure  that  real-world  situations  can  be  represented  in 
the  model: 

1.  In  a  probabilistic  decision  type,  the  next  task  to  execute  is  determined  by  the  relative 
probabilities  of  all  tasks  listed.  Probabilistic  decisions  allow  only  one  of  the  following  tasks  to 
execute. 
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2.  In  a  multiple  decision  type,  all  the  tasks  with  conditions  that  evaluate  to  non-zero  will 
execute.  This  allows  for  one  or  more  tasks  to  begin  execution,  based  on  rules  that  determine 
execution  tasks. 

3.  In  a  tactical  decision  type,  the  next  task  to  execute  is  the  task  with  the  condition  that 
evaluates  to  the  highest  value.  This  allows  for  rule-based  decisions.  A  tactical  decision  type 
differs  from  the  multiple  type  in  that  only  one  following  task  is  executed. 

Variables  and  algebraic  expressions  can  be  used  in  the  branching  logic,  and  the  value  of 
the  variables  can  be  changed  by  conditions  in  the  model.  This  allows  complete  control  and 
manipulation  of  the  network  flow. 

Simulation  Clock 

The  simulation  clock  tracks  the  simulated  time  as  the  model  executes.  Time  can  be 
advanced  in  the  simulation  either  in  fixed  or  variable  time  intervals.  In  a fixed  interval 
simulation,  the  simulation  clock  is  advanced  in  fixed  time  intervals;  the  simulation  is  referred  to 
as  clock  driven.  Examples  of  clock-driven  simulations  are  chemical  processes  and  weather 
models.  In  a  variable  interval  simulation,  events  are  used  to  advance  the  clock  in  initial  value 
and  type  (integer,  real,  array  of  integers,  array  of  real  numbers). 

Expressions 

An  expression  can  be  a  calculation,  formula,  function,  or  statement  that  supplies  a  value 
or  performs  an  operation.  Expressions  are  used  to  supply  numerical  values  such  as  mean  times 
or  true  or  false  values  such  as  those  used  in  release  conditions.  They  are  used  to  make  changes 
in  the  state  of  the  model,  such  as  beginning  effects  and  ending  effects.  Each  expression  in 
MicroSaint™  must  end  with  a  semi-colon  and  can  include  any  of  the  following  elements: 

•  Constants 

•  Variables 

•  Functions  (groups  of  expressions  that  can  be  referred  to  or  called) 

•  Comments 
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•  Mathematical  operators  (+,  *,  /, A,  %,  ( )) 

•  Assignment  operator  (:=) 

•  Adjustment  operators  (+=,  -=,  *=,  /=) 

•  Logical  operators  (>,  >=,  <,  <=,  &,  ==,  I,  <>) 

•  If-then-else  and  while-do  statements 

Scenario  Event 

A  scenario  event  is  scheduled  to  occur  at  a  specific  time  (in  simulation  time)  during 
model  execution.  Scenario  events  are  also  used  to  change  variable  values,  thereby  changing  the 
state  of  the  model.  These  can  be  one-time  events  or  they  can  repeat  at  regular  intervals.  An 
example  of  a  one-time  event  would  be  setting  a  variable  at  simulation  time  zero,  indicating  the 
number  of  alarms  that  will  sound  during  a  nuclear  plant  disturbance.  Scenario  events  are  defined 
by  supplying  the  following  information  for  each  event  in  the  event  description  dialog  box: 

1.  Time  of  occurrence. 

2.  Whether  the  event  should  repeat  and  at  what  interval. 

3.  Time  when  you  want  the  event  to  stop  repeating,  if  applicable. 

4.  The  expressions  you  want  executed  at  the  specified  time(s). 

Model  Execution 

When  the  model  execution  is  started,  an  entity  begins  at  the  first  task  node  in  the  model. 
If  the  release  condition  for  that  task  is  evaluated  to  “true,”  then  the  task  executes.  The  effect(s) 
that  the  task  has  on  the  system  are  evaluated,  based  on  the  expressions  defining  the  task 
description.  The  changes  are  expressed  in  variables  that  can  be  used  in  other  tasks  in  the  model. 
Once  the  task  is  completed,  the  entity  proceeds  to  the  next  task  in  the  network  diagram.  When 
more  than  one  path  is  available,  the  branching  logic  is  used  to  determine  the  path  the  entity  will 
follow.  In  general,  the  entire  network  diagram  is  traversed  by  the  entity  and  the  model  is 
completed  when  the  entity  reaches  the  end  of  the  last  task  in  the  network.  Models  can  have 
conditions  that  send  entities  through  the  network  until  a  specified  simulation  time  or  until  a  pre¬ 
determined  number  has  completed  the  simulation. 
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Data  Collection  During  Model  Execution 

The  output  data  for  a  simulation  are  specific  values  of  model  variables  recorded  at 
specific  times  during  the  execution  of  the  model.  The  data  recorded  are  used  to  answer  the 
questions  about  the  system  being  modeled.  The  output  is  similar  to  the  results  of  an  experiment. 
Data  output  can  include  measures  of  system  effectiveness  or  can  be  used  for  system  diagnostics. 
Some  examples  of  useful  output  are  resource  use,  cost,  and  errors  initiated. 

Data  are  collected  during  the  execution  of  a  MicroSaint™  model  using  a  feature  called 
“snapshots”.  Snapshots  provide  a  way  to  collect  values  of  variables  at  specified  points  during 
model  execution.  They  can  be  programmed  to  occur  at  specific  clock  times,  when  a  task  begins 
or  ends,  or  when  a  model  execution  ends.  Snapshots  are  defined  by  providing  the  following 
information  in  the  snapshot  description  dialog  box: 

1 .  A  name  for  the  document  where  the  data  are  stored. 

2.  The  “trigger  types”  for  the  snapshot  (end  of  run,  clock,  begin  task,  end  task). 

3.  The  number  of  the  triggering  task,  if  applicable. 

4.  The  start  time,  stop  time,  and  repeat  interval,  as  applicable,  if  the  snapshot  has  a  clock 

trigger. 

5.  The  names  of  the  variables  for  which  you  want  to  record  values. 

Once  the  snapshots  have  been  defined,  they  can  be  set  to  “on”  or  “off’  during  model 
execution.  When  they  are  turned  “on,”  the  variable  values  are  stored  in  a  results  file  with  the 
extension  “.res”.  After  the  file  is  opened,  the  analyze  commands  in  MicroSaint  can  be  used  to 
generate  statistics  and  create  graphs  from  the  data.  The  data  can  also  be  imported  into  other 
statistical  analysis  packages. 
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1 3.  ABSTRACT  (Maximum  200  words) 

The  purpose  of  the  research  project  was  to  understand  the  future  crew  environments  for  developing  unmanned  aerial  vehicle 
(UAV)  systems.  A  variety  of  human  engineering  tools  (job  assessment  software  system  [JASS],  enhanced  computer-aided 
testing  [EC AT],  and  MicroSaint™)  were  used  to  address  crew  issues  related  to  the  utility  of  having  rated  aviators  as  crew 
members,  supplementing  current  crews  with  imagery  and  intelligence  specialists,  and  the  use  of  automation  to  improve  systems 
efficiency.  Data  from  70  soldiers  and  experts  from  Fort  Huachuca,  Arizona,  Fort  Hood,  Texas,  and  Hondo,  Texas,  were 
collected  as  part  of  this  effort.  The  general  finding  was  that  the  use  of  cognitive  methods  and  computerized  tool  sets  to 
understand  future  crew  environments  proved  to  be  cost  effective  and  useful.  Specifically,  no  evidence  was  found  to  support  a 
requirement  for  rated  aviators  in  future  Army  missions,  but  the  use  of  cognitively  oriented  embedded  training  simulators  was 
suggested  to  aid  novices  in  developing  the  cognitive  skills  evinced  by  experts.  The  efficacy  of  adding  imagery  specialists  to 
96U  crews  was  discussed,  and  specific  recommendations  related  to  automation  were  derived  from  the  workload  modeling. 
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